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Summary 

This  paper  presents  the  findings  of  a  survey  carried  out  during  2001/2002  to  identify  Human  Factors  issues 
and  concerns  associated  with  in-service  equipment.  Training  was  one  of  the  topics  under  investigation  and 
15  of  the  projects  interviewed  out  of  a  total  of  24  projects  highlighted  specific  training  related  concerns.  The 
types  of  issues  found  were  categorised  as  follows: 

•  Degree  of  trainer  fidelity 

•  Maintainer  training  aids 

•  Trainer  utility/utilisation 

•  Documentation  support 

•  Training  gaps/confiicts 

•  Scenario  definitions 

The  paper  will  discuss  the  specific  issues  to  determine  which  ones  present  future  challenges  in  terms  of 
training  needs 

Introduction 

A  2-year  study  was  carried  out  between  2000  and  2002  on  behalf  of  QinetiQ  CHS  for  UK  MOD  to 
investigate  the  status  of  HFI  application  in  UK  defence  equipment  procurement  Management  within  the 
Defence  Procurement  Agency  (DPA)  and  Defence  Eogistic  Organisation  (DEO)  Integrated  Project  Teams. 
The  overall  aim  of  the  study  was  to  facilitate  the  allocation  of  HFI  research  budgets  within  the  UK  MOD 
Corporate  Research  Programme 

The  first  year  of  the  study  investigated  specifically  the  methods,  techniques  and  organisational  aspects 
associated  with  the  planning  of  HFI  within  the  IPTs.  The  2"‘*  year  addressed  HF  issues  associated  with  In- 
Service  Equipment  and  it  is  the  findings  related  to  training  matters  of  the  2”‘*  year  of  the  latter  that  are 
presented  in  this  paper 

The  structure  of  the  paper  is  as  follows: 

•  Our  approach 

•  Project  Selection 

•  Taxonomy  of  HF  Issues 

•  Overall  findings 

•  Training  Issues 

The  projects  interviewed  will  not  be  named  for  military  and  commercial  confidentiality  reasons. 


Paper  presented  at  the  RTO  HFM  Symposium  on  “Advanced  Technologies  for  Military  Training", 
held  in  Genoa,  Italy,  13  -  15  October  2003,  and  published  in  RTO-MP-HFM-101. 
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Approach  Summary 

In  order  to  scope  the  project  within  the  available  budget,  careful  selection  of  projects  had  to  be  made  to 
provide  a  representative  sample  of  the  platforms  and  equipment  in  service.  This  was  carried  out  using  a  4- 
dimensional  criteria  space  covering  project  capability,  size,  time  in  service  and  level  of  procurement  control. 
A  semi-structured  interview  technique  was  determined  to  be  appropriate  for  the  assessment  and  a  generic 
checklist  was  developed  covering  the  full  range  of  Human  Factors  topics.  Each  IPT  received  an  outline  of 
the  topics  that  were  going  to  be  addressed  during  the  interviews  to  ensure  that  the  appropriate  team  members 
were  available  and  fully  briefed.  The  interviews  were  conducted  with  the  aid  of  a  recorder  and  the  summary 
reports  were  sent  to  each  IPT  prior  to  the  analysis  being  carried  out.  Further  validation  visits  at  service  bases 
were  carried  out  for  about  a  third  of  the  projects. 

Project  Selection 

The  down  selection  of  the  projects  was  based  upon  4  criteria  that  were  deemed  to  be  significant  following 
the  first  year  of  the  study: 

•  Capability  Area: 

o  Strategic  Deployment 
o  Manoeuvre 
o  Strike 

o  Information  Superiority 

•  Project  Size: 

o  Large  -  A,  B(>£100M) 
o  Small  -C,D(<£100M) 

•  Length  of  Time  in  Service: 

o  Mature  -  Post  Mid  life  upgrade  stage/long  time  in  Service 
o  Immature  -  Relatively  short  length  of  time  in  Service/Pre  upgrade 

•  Level  of  Procurement  Control 

o  High  -  UK  bespoke  systems/high  degree  of  specification  control 

o  Low  Control  -  COTS,  MOTS,  upgrade  of  existing  systems,  international  collaborative 
projects 

It  was  recognised  that  the  selection  of  24  projects  meant  that  there  could  not  be  a  balanced  sample  of  every 
criteria  combination  but  it  was  agreed  that  there  should  be  at  least  a  split  between  high  and  low  control  and 
across  all  capability  areas.  The  project  selection  within  these  categories  is  shown  in  table  1  below: 


Strategic 

Deployment 

(7) 

Manoeuvre 

(6) 

Strike 

(6) 

Information 

Superiority 

(5) 

Size 

15  Large 

9  Small 

4  Large 

3  Small 

4  Large 

2  Small 

3  Large 

3  Small 

4  Large 

1  Small 

Maturity 

1 0  Mature 

14  Immature 

5  Mature 

2  Immature 

2  Mature 

4  Immature 

2  Mature 

4  Immature 

1  Mature 

4  Immature 

Control 

17  High 

7  Low 

5  High 

2  Low 

4  High 

2  Low 

5  High 

1  Low 

3  High 

2  Low 

Table  1  Summary  of  Numbers  of  Projects  by  Category 
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Taxonomy  of  Generic  HF  Issues 

A  checklist  of  HF  issues  was  derived  from  a  set  of  technical  topics  that  had  been  successfully  used  on 
maritime  projects  (Sea  System  Publication  SSIO)  and  formed  a  good  basis  for  discussion  with  the  IPTs.  The 
topics  were: 

•  Operational  -  issues  associated  with  operational  procedures,  role  or  scenario  changes,  task 
allocation  etc 

•  Organisation  (Manpower)  -  issues  associated  with  manning  (operation  and  support)  and 
communication  issues  within  and  between  teams  etc 

•  Personnel  -  issues  associated  with  user  characteristics,  skills,  gender  etc 

•  Training  -  issues  associated  with  the  effectiveness  of  training 

•  Operability  -  issues  associated  with  the  Human-Machine  Interface  affecting  overall  mission 
effectiveness 

•  Layout  -  issues  associated  with  the  physical  layout  of  the  working  environment 

•  Environment  -  habitability  issues  (noise,  heating,  ventilation,  lighting  etc)  that  may  affect  overall 
human  performance 

•  Maintenance  -  issues  associated  with  equipment  maintainability  and  availability 

•  System  Safety  -  concerns  about  accidental  damage  to  the  user 

•  Health  Hazard  -  concerns  about  human  exposure  to  hazardous  or  unhealthy  environments 

In  addition  interviewees  were  invited  to  outline  any  additional  topics/issues  specific  to  the  project/equipment 
that  were  not  covered  adequately  by  the  topics/issues  listed.  The  number  and  type  of  concerns  raised 
reflected  the  knowledge  and  background  of  the  interviewees  and  did  not  necessarily  cover  all  of  the  topic 
areas.  As  most  of  the  interviews  were  conducted  within  the  Defence  Logistics  Organisation,  where 
maintaining  and  sustaining  capability  is  their  primary  focus,  the  concerns  tended  to  be  more  support  relate 


Overall  Findings 

The  issues  were  abstracted  from  the  interviews  and  categorised  within  the  10  topic  areas.  Each  topic  area 
was  further  sub-categorised  into  root  problem  areas. 

There  were  in  total  about  400  different  concerns  raised  and  the  overall  findings  are  shown  here: 
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Figure  1  Number  of  Concerns  within  Topic  areas 
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Each  topic  area  was  analysed  against  the  highest  number  of  reports  of  specific  issues  within  each  sub¬ 
category  and  table  2  below  shows  the  top  3  issues  within  each  of  the  topic  areas: 


Technical  areas 

Highest  Issue 
and  No  of 

occurrences 

2"“  Highest 

Issue  and  No  of 
Occurrences 

Highest 

Issue  and  No  of 

occurrences 

%  of  top  3 

Issues 

Addressed 

Operational 

Role  Change 

11 

Environment 

Change 

5 

Tactical  Capability 

5 

62% 

Organisation/Manpower 

Comms  Capability 

14 

Comms  Quality 

7 

Human 

Performance 

7 

68% 

Personnel 

Personnel 

Availability 

16 

Skill  Eevel 

8 

Physical 

Constraints 

5 

66% 

Training 

Trainer 

Representation 

6 

Maintainer 
Training  Aids 

5 

Trainer 

Utility/Utilisation 

5 

25% 

Operability 

Control  Operation 

8 

Display 

Viewability 

6 

Tactical  Capability 

5 

58% 

Layout 

Workspace 

9 

Kit/clothing 

5 

Operability 

Compromised 

4 

22% 

Environment 

Temperature 

14 

Noise 

10 

Ventilation 

5 

28% 

Maintenance 

Maintainance 

Access 

11 

Availability  of 
Spares 

8 

Workload 

7 

27% 

System  Safety 

Unsafe  Loads 

6 

Physical  Accident 
Risk 

4 

Atmospheric  Risks 

4 

86% 

Health  Hazard 

High  Risk 
Conditions 

8 

Toxic  Substances 

6 

Sensory 

Degradation 

5 

68% 

Additional  Topics 

Survivability 

6 

Vulnerability 

3 

Anthropometric 
Data  Shortage 

1 

40% 

Additional  Issues 

Operational 

Effectiveness 

4 

Can  Do  Attitude 

3 

Configuration 

Control 

1 

38% 

Table  2  Summary  of  most  frequent  issues  within  each  technical  topic 

The  final  column  shows  the  percentage  of  concerns  that  were  likely  to  be  addressed  during  the  life  of  the 
specific  programmes.  It  can  be  seen  from  the  table  that  the  training  concerns  were  one  of  the  areas  least 
likely  to  be  addressed. 
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15  out  of  24  of  the  projects  highlighted  specific  concerns  associated  with  training  and  the  main  areas  of 
concerns  were  as  follows: 

•  Trainer  representation 

•  Maintainer  training  Aids 

•  Trainer  Utility/utilisation 

•  Documentation  support 

•  Training  gaps  and  conflicts  in  training 

•  Scenario  evolution 

Together  training  facilities  excluding  documentation  support  amounted  to  over  two  thirds  of  the  training 
issues  raised.  The  diagram  below  shows  the  percentage  of  issues  raised  within  each  sub-category 


Scenario  Evolution 
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Figure  2  Relevant  Training  Issues 


Trainer  Representation 

This  area  of  concern  was  identified  by  6  of  the  projects  interviewed.  Many  of  the  training  facilities  appeared 
to  be  lacking  in  their  operational  representation  and  this  increases  the  need  for  on-job  training.  The 
equipment  was  not  flexible  enough  to  give  realistic  training  Thus  in  some  cases  operational  readiness  only 
comes  from  expensive  training  exercises  at  sea  or  where  the  fidelity  is  lacking,  live  firings  on  ranges.  There 
were  particular  issues  associated  with  older  programmes  where  enhanced  capabilities  were  being  provided  to 
the  platforms  but  not  to  the  trainers  in  the  same  timescales.  The  view  from  the  projects  interviewed  was 
‘Front  Line  First’  ie  get  the  capability  to  the  front  line  and  let  the  training  catch  up.  In  some  cases  but  not  all, 
the  lack  of  fidelity  did  have  operational  repercussions.  The  lack  of  visual  or  motion  cues  on  the  simulators 
also  reduced  the  effectiveness  of  the  training. 

Maintainer  Training  Aids 

5  of  the  projects  interviewed  cited  poor  training  aids  for  the  support  staff.  In  some  cases  there  were  no 
trainers  for  the  maintainers  leaving  all  the  skills  to  be  acquired  via  on-job  training.  Generally  the  equipment 
care  training  was  ineffective  and  one  IPT  provided  photographic  evidence  of  complete  lack  of  equipment 
care  -  e.g.  optics  cleaned  with  a  wire  brush.  Another  outcome  of  poor  maintainer  training  was  the  tendency 
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to  change  LRUs  without  proper  fault  diagnosis.  This  was  largely  attributed  to  a  complex  system  where  the 
technicians  lacked  adequate  knowledge  and  skills  to  make  the  correct  diagnosis. 

Trainer  Utility/utilisation 

There  were  significant  issues  associated  with  trainer  utility  with  5  IPTs  highlighting  issues  associated  with 
the  effectiveness  of  their  training  aids.  In  addition  the  use  of  the  simulators  to  gain  a  much  deeper 
understanding  of  the  capabilities  of  the  system  potentially  provided  a  valuable  tactics  development  tool  but 
poor  utilisation  meant  that  they  were  not  making  the  best  use  of  a  highly  complex  and  capable  system.  There 
were  no  airborne  multiple  crew  simulators  for  the  projects  interviewed  that  enabled  multi-crew  platforms  to 
train  and  rehearse  effectively.  However  the  naval  platforms  did  have  good  training  facilities,  which  were 
used  very  effectively.  In  addition  the  operational  load  and  staff  shortages  made  dedicated  training  time 
limited  but  generally  on-job  training  was  found  to  be  effective. 

Documentation  support 

3  of  the  projects  interviewed  highlighted  problems  associated  with  documentation  support.  The  type  of 
problems  encountered  varied  between  the  obsolescence  of  the  documentation  media  and  total  absence  of  any 
training  material.  In  one  particular  case  the  media  was  a  micro-fiche  reader  of  which  insufficient  were 
available.  Thus  there  was  a  high  dependability  on  paper  versions  that  were  not  always  up  to  date.  This  had  a 
particular  impact  upon  maintenance  training.  In  another  case  there  was  a  complete  lack  of  training  material 
for  the  training  support  staff.  Therefore  they  had  to  spend  time  with  the  manufacturers  familiarising  with  the 
kit. 

Training  gaps  and  conflicts  in  training 

The  issues  associated  with  training  gaps  and  training  conflicts  were  all  associated  with  vehicle  systems 
where  drivers  were  both  operators  and  maintainers.  The  training  for  drivers  and  maintainers  was  carried  out 
at  different  establishments  and  this  occasionally  gave  rise  to  conflicts  and  overlaps  between  the  respective 
manuals  that  lay  down  crew  maintenance  and  maintainer  tasks.  Doctrine  and  training  occasionally  had  to  be 
updated  after  incorrect  use  of  procedures.  Refresher  training  was  highlighted  as  a  problem  by  one  IPT  as 
each  regiment  has  only  one  suite  of  gunnery  simulators  to  serve  4  squadron  of  14  crews.  The  amount  of 
dismounted  ‘other  duties’  often  led  to  long  gaps  without  refresher  training  which  could  lead  to  ‘skill  fade’. 

Scenario  evolution 

In  one  particular  example  the  scenarios  on  the  simulator  were  difficult  to  change  and  the  technicians  lacked 
the  skill  or  training  to  make  the  necessary  changes  thus  once  the  scenarios  become  predictable  the  training 
becomes  ineffective.  This  directly  impacted  upon  simulator  utilisation  as  discussed  above 


Impact  of  Type  of  Project 

As  discussed  previously  the  projects  were  selected  against  4  criteria  to  determine  whether  the  capability  area, 
size  of  project,  maturity  of  project  and  level  of  control  influenced  the  numbers  or  types  of  concerns.  The 
results  are  shown  in  the  diagram  below.  The  percentage  bars  show  each  category  normalised  as  a  percentage 
of  the  total  number  interviewed  within  that  category  in  order  to  take  account  of  distribution  differences. 
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Figure  3  Summary  of  Issues  against  the  project  characteristics 


This  figure  shows  no  significant  differences  in  the  number  of  training  concerns  across  the  4  procurement 
areas  but  it  does  show  that  the  majority  of  issues  were  those  associated  with  large  programmes  within  a  pre¬ 
upgrade  phase.  The  type  of  problem  exhibited  for  immature  programmes  are  mainly  fidelity  related.  In 
addition  the  bespoke  type  systems  (high  control)  yielded  more  problems  than  the  COTS  type  systems  (low 
control). 


Conclusions 

•  Cost  factors  are  driving  the  need  for  increased  use  of  virtual  training  solutions.  However  the  findings 
from  the  survey  revealed  that  there  is  still  a  reliance  on  field-based  training  due  to  fidelity  issues.  These 
fidelity  issues  were  affecting  the  utilisation  of  the  simulators  to  the  detriment  of  maximising  the 
effectiveness  of  the  systems. 

•  Achieving  the  right  balance  between  simulator  and  field-based  training  continues  to  cause  concern. 
Whilst  the  technology  to  create  powerful  graphic  representations  to  give  good  visual  feedback  is 
available  now  and  the  processing  capability  enables  this  to  be  carried  out  in  real  time,  it  is  likely  that 
there  will  always  be  aspects  of  the  task  that  cannot  be  trained  in  a  simulator. 

•  Front  Line  First  philosophy  means  that  the  upgrade  to  training  devices  may  not  provide  the  lead  training 
time  required.  This  could  impact  the  operational  effectiveness  of  the  delivered  capability. 

•  The  advancement  of  highly  complex  IT  systems  is  impacting  upon  selection  and  training  of  new  skill 
areas.  This  may  require  novel  approaches  to  training  to  acquire  the  level  of  knowledge  and 
understanding  of  these  complex  systems. 

•  Lack  of  maintenance  training  aids  is  leading  to  ineffective  and  inefficient  support  to  the  front  line.  The 
focus  on  equipment  care  needs  to  be  given  more  prominence. 
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•  Maintenance  of  highly  complex  IT  system  trainers  currently  requires  specialist  technicians  to  ensure  the 
systems  can  be  fully  exercised  within  the  full  range  of  threat  scenarios.  There  are  retention  problems  for 
these  skill  areas  as  there  are  lucrative  contracts  available  within  industry. 


Recommendations 

•  The  fidelity  of  the  training  device  must  not  be  compromised  as  this  could  result  in  negative  training 
problems  and  impact  upon  operational  effectiveness 

•  There  needs  to  be  greater  consideration  of  embedded  ,  on-job  training  facilities  to  compensate  for 
shortage  of  personnel  and  off-job  training  time. 

•  Rapid  augmentation  of  new  capabilities  into  the  training  aids  needs  to  be  considered  in  the  design  of  the 
simulators  to  ensure  that  they  maintain  the  required  lead  training  time. 

•  There  needs  to  be  much  more  emphasis  on  maintenance  training  and  equipment  care  as  this  clearly 
impacts  upon  operational  readiness  of  the  equipment  and  platforms. 

•  The  training  of  highly  complex,  multi  modal  systems  needs  further  research  to  enable  the  level  of  mode 
awareness  to  be  acquired  although  quite  clearly  the  source  of  this  problem  is  a  design  issue. 

•  Maintenance  of  simulators  needs  to  be  simplified  to  ensure  ease  of  scenario  update  to  enable  full 
utilisation  of  the  systems  across  a  full  range  of  scenarios. 
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